The Transmission Control Protocol (TCP) provides the backbone of the modern Internet. TCP is used to provide ordered delivery of data between two hosts using a stateful connection. TCP assures that one host can send a bytestream (e.g., file, message, etc.) over an unreliable communication medium (unreliable in the sense that delivery is not guaranteed) as a series of individual packets, and that the remote host will be able to reconstruct the bytestream in the correct order and with all the data intact. To achieve this reliability, both hosts participate in the delivery; the sender provides a “sequence number” in each packet it transmits, which increases as each packet is transmitted. The sequence number is increased by an amount corresponding to the number of bytes in the payload of each packet. This approach allows packets to be reassembled correctly, even if they are received out of order.
The recipient is responsible for acknowledging each byte of data it receives using a cumulative acknowledgement scheme. Specifically, the recipient transmits an “acknowledgement number” to the sender specifying the number of the next byte expected to be received. This also signifies to the sender that the recipient has received all transmitted bytes up to that indicated by the acknowledgment number. For example, if a sender sends a packet containing four payload bytes with a sequence number field of 10000, then the sequence numbers of the four payload bytes are 10000, 10001, 10002 and 10003. When this packet is successfully received, the recipient would send back an acknowledgment number of 10004, since 10004 is the sequence number of the next byte it expects to receive in the next packet.
By monitoring the acknowledgements coming back from the recipient, the sender can determine when one or more packets have been lost in transit and need to be retransmitted. On top of this straightforward reliability mechanism, TCP also implements several other important features such as flow control and congestion avoidance.
The reliable transmission features of TCP limit its flexibility. Since every packet of data is accounted for using acknowledgements, an intermediate network device is unable to modify the bytestream by changing individual packets; any addition or removal of data disrupts the sequence number of the bytes received by the recipient, and causes a change in the acknowledgement numbers sent by the recipient. These changed acknowledgements no longer correspond directly to the bytes of data the sender expects to receive, leading to instability, unnecessary retransmissions, degradation of network throughput as congestion controls are enacted, and in some cases a complete breakdown of the connection.
Therefore, if an intermediate network device is to modify the data exchanged between two hosts, the intermediate device generally divides the connection into two separate TCP connections and coordinates data transmission between the two hosts. This practice is known as “proxying” a connection, and the intermediate device is known as a “TCP proxy”.
Many modern web technologies employ a TCP proxy. A classic example of a TCP proxy is a Hypertext Transfer Protocol (HTTP) gateway, which is positioned between a web browser client and web server to improve performance by caching content, modifying requests to use more efficient delivery mechanisms, and the like. The gateway does not allow a direct connection to be established between the client and server, instead the gateway establishes two separate connections and acts as a TCP proxy.
Conventional HTTP gateways are generally “transparent” (sometimes called “forced” or “intercepting”), meaning that every connection attempted through the gateway will be intercepted and split into two connections, but that the client and server need not be aware of the proxy. On busy networks, proxying a large plurality of TCP connections is very resource intensive, making proper provisioning difficult.